TDIing out loud, ok SDIing as well

As Eisenhower said, Plans are Useless

2021-10-04T20:21:00.006+02:00

...as soon as the first shot is fired. However, the art of planning is invaluable. In other words, plan for change and be ready to change your plans. Repeatedly. This be the foundation of an agile approach to solution development.I agree with Ken Schwaber in that many agile methodologies do not feel very agile. Some major contenders are thinly veiled command-and-control frameworks in which you

Agile is as Agile does...

2020-12-13T11:43:00.000+01:00

...to paraphrase Forest Gump. TDI (aka SDI) is an extraordinarily agile platform for exploration, prototyping and crafting enterprise-strength solutions for data movement and services. However, using it or any other tool does not an agile developer make. Agile development is all about mindset and culture rather than processes and tools. In this age of industrialized and weaponized Agile (the noun

How to turn security events from QRadar into self-healing actions on infrastructure

2020-12-01T14:04:00.000+01:00

Here for your perusal is a self-contained solution that provides a service AL that listens for incoming requests coming from QRadar Event Rules. You write your own ALs to perform actions based on various events, using data shared by QRadar in the event action request JSON. Note that this is part of the Guardium bundle, leveraging it to automatically perform reconfiguration, like locking

SDI, TDI, what's in a name

2020-01-23T09:21:00.002+01:00

When IBM acquired the integration powertool back in 2002 it was called 'IBM Data Integrator'. Then after a bit of political squabbling, it was moved to the Tivoli brand and renamed 'Tivoli Directory Integrator'. Now, after its move to the Security brand a few years back, the name became 'Security Directory Integrator'. I just think of it as 'Integrator', although my fingers still type 'TDI'. Old

How To Ask Smart Questions

2019-06-17T14:37:00.005+02:00

Although this article is particularly about asking hackers there are some universally relevant pearls here that are ring particularly true today: http://catb.org/~esr/faqs/smart-questions.html It's easiest to go for the quick fix when it comes to information gathering. It's apparent in the alarming number of people who's primary news source is social media, and it's highlighted nicely in this

SDI/TDI as a Windows Service

2019-06-04T14:22:00.002+02:00

NOTE: You must remember to copy the ibmdiservice.exe and ibmdiservice.props files from the \win32_service folder to your Solution Directory (soldir) before you run the service installer. It is a common mistake to run the service installer from the wrong folder - i.e. not your soldir. Often this is done from \win32_service where the .exe and .props files are found. And if

Thinking about agility

2018-09-25T16:08:00.002+02:00

My favorite agile tool is TDI (and please forgive an old man for having trouble shifting to 'SDI'). Not only can I whip together integration service prototypes faster than anything I can do in Java (Spring), or Python or TurboPascal, but I can also build Ops features this way. Monitoring and remote control (pause, restart, failover/failback) for enabling auto-healing, or at least making the

Collecting my ramblings on Connector Loops in one place

2017-10-24T11:51:00.001+02:00

November 7, 2013 - Connector Loops and why you're gonna love them September 30, 2013 - Why Loops Instead Of Lookup Mode November 7, 2013 - Why Connector Loop June 8, 2010 Loop a Connector today

TDI Getting Started and Ensuring Basic Understanding

2017-09-01T01:14:00.000+02:00

The Basics Install TDI (https://ibm.biz/BdjsLw) locally to your laptop. This lets you customize your TDI environment easier. If you can't get a licensed version, start with the free version (https://ibm.biz/BdjiMg). It's fully functionally, will give you time needed to get started solving with TDI, and the license file can be replaced when you get yours. Do NOT install under Program Files on

How to make the Search/Reconcile AL for an RMI Adapter against a REST or WS API

2016-12-13T14:25:00.003+01:00

Writing RMI Adapter AssemblyLines can be tricky at times, especially when it comes to the Search AL - the one used for Reconcile operations. This is because the Dispatcher calls the Search AL in Cycle mode, which causes it to work differently than you might expect. The usual behavior of an AL is that an Iterator in the Feed section delivers data - one Entry at a time - into the Data Flow

Importing a client certificate in order to access an https URL

2016-12-12T12:47:00.000+01:00

When you go to make an HTTP request to a service that requires an SSL connection, you'll get an exception that your HTTP Client Connector cannot set up the secure link to the service. Unlike your browser, TDI (SDI) does not automatically import the client certificate. You have to do this yourself, and this post is about how to do this. For this example, we'll try to access https://

JSON and XML Tutorial - Part 6, Making a custom Form

2016-11-21T14:29:00.001+01:00

As promised, this post will walk you through creating a custom Connection Tab Form for the Connector scripted in the previous post. Myself being in true form, I will begin with a promised explanation and then a bit of theory. Internal name of a component In my last post I promised to show you how to discover the internal name of a component. Otherwise, how can you know that the HTTP Client

JSON and XML Tutorial - Part 5, Scripting a Connector

2016-11-17T15:23:00.000+01:00

Continuing from my last rambling post, we'll script an Iterator Connector that will return the same parsed entries as our Conditional Loop did. I'll begin with a little theory, which you will find duplicated other places like the How-To page of www.tdi-users.org and this venerable video tutorial on the subject. To script a component in TDI (SDI) you start with the basic Script version of that

JSON and XML Tutorial - Part 4, JSON example

2016-11-16T15:36:00.002+01:00

Practice examples are helpful when mastering new techniques. In this example we'll make a REST request and then handle the return JSON payload. We'll be using the service available at the following URL: http://www.ideal-status.nl/static/consumer_notification_advice.json If you dial up this URL you'll get JSON that looks like this: {"0021":{"issuer_id":"0021","issuer_bic":"RABONL2U","

CouchDB/Cloudant Connector

2016-11-14T15:46:00.002+01:00

Those of you who want to learn more about the truly marvellous concept of MapReduce (thanks, Google!) or just want to play with the can't-believe-it's-free CouchDB, here is a connector I wrote a while back that supports both CouchDB and Cloudant. All you need to do is download a free copy of CouchDB (painless install and loads of fun), or sign up for your free Cloudant db in the cloud at https://

JSON and XML Tutorial - Part 3, XML example

2016-02-22T12:13:00.000+01:00

Continuing on from Part 1 of this tutorial, let's play a little with XML data using the HEntry techniques discussed. Here is some example XML that we'll play with: Direwolf Robb Bran Lion

Externalizing an Attribute Map to a file

2016-01-13T18:43:00.000+01:00

Hit the More... button at the top of the AttMap - to the right of the Add and Delete buttons - and select Change inheritance. In the resulting dialog click External attribute map and select a file. The external mapping feature of TDI uses text files that contain one or more mapping rules. Each mapping rule looks like this: name=assignment. Here is an example of a mapping rule: uid= This

What is the System Store?

2015-04-10T15:55:00.000+02:00

The System Store is a database that TDI uses to 'remember' stuff like: the last change read, when you use one of the Change Detection Connectors (also called CDCs); the snapshots of data used to compute changes when using the Delta Engine together with an Iterator Connector; data read using the System Store Connector, which writes and reads full Entry objects with any number of Attributes;

Importing a client certificate from an https service

2015-03-02T19:52:00.002+01:00

I have cobbled together a simple AL that can be run from the commandline in order to import a client certificate from an HTTPS service. For example: ibmdisrv -c configs/tdiingOL.xml -r importcert -0 https://supportcenter.checkpoint.com The above call results in this output (including the standard TDI is startup messages): CTGDKD024I Remote API successfully started on port:1099, bound to:'

JSON and XML Tutorial - Part 2

2015-02-11T14:21:00.004+01:00

Well, this one is focused on JSON and I rolled it into a quick video. https://www.youtube.com/watch?v=VHmf76FMI-U Enjoy, and please comment if anything is unclear (or unsaid).

Easily implementing a monitoring API to your TDI solution

2015-02-05T13:55:00.000+01:00

All you have to do is make AssemblyLines with the names of the operations you want to expose. For example, I have a solution that catches incoming events over TCP and dispatches the payload data to one or more 'event triggered' AssemblyLines. Like all server mode based ALs, this one uses multiple threads to deal with client traffic (the AL Pool), and shared log output can get a little

Passing arguments to your AL when launching it from the command-line

2015-01-26T12:25:00.001+01:00

In the ibmdisrv command-line used to launch your AssemblyLine, you can use one of the user-defineable options -0 through -9 to pass information into the AL. For example, in order to pass the name of an input file to the AssemblyLine named 'ReadFile' you could use this command-line: ibmdisrv -c MyConfig.xml -r ReadFile -0 c:/Input/HRExport.csv Then in the Prolog - Before Initialize Hook of

Hardening an AssemblyLine

2014-11-24T13:27:00.000+01:00

This is done with a little effort in three areas: Error handling Log handling Auto-reconnect For Error Handling you need to add code to the Prolog - On Error Hook for connection errors in components, and to the DataFlow - Default On Error (the catch-all Error Hook for all Connector modes). Here you will want to capture all error information to the log in order to help troubleshooting the issue.

Null Behavior

2014-08-04T11:49:00.004+02:00

I've gotten this question (again) and decided to explain it here so that Google can find it. Null Behavior allows you to deal with missing data without having to write Javascript. The Null Behavior feature lets you define what a 'null' attribute is and how it should be handled. By default the definition of 'null' is that the source attribute is missing or has no value (null value). The

mapReduce Revealed!

2014-07-12T11:25:00.000+02:00

Here is an excellent explanation of mapReduce that I've had to share (again and again). Enjoy! http://www.slideshare.net/okurow/couchdb-mapreduce-13321353